In the ideal world, a system will be created correctly on its first iteration without mistakes and cater for all eventualities. In the real world, mistakes and unexpected events happen.
This is certainly the case in computer software. It is not unusual in complex systems for components and routines to be written or created by different individuals or organisations. Indeed, it is not at all unusual to inherit what is termed ‘legacy’ code as part of a project; company buy-out; or merger that you are expected to support and continue to refine even though you had no input in its design or implementation.
Debugging is a problem faced by developers worldwide. They have a problem in their code. They have pinpointed the outward manifestation of the problem, using a dynamic analysis tool, but they have no clue as to the root cause. The debugging to find the root cause of the problem (necessary to effect a fix) is often long and protracted. On large poorly maintained code bases the debug time can run into weeks. Debugging techniques often rely heavily on manual insertion of trace statements to show exactly what the code is doing.
Many compilers and software development systems offer assistance to a developer as software is being written to attempt to identify syntactic errors at design-time. This is generally referred to as static analysis. However, once a project reaches a stage where code is compiled and run, the help available diminishes significantly. At this stage, dynamic testing typically takes the place of static testing techniques.
Dynamic testing (or dynamic analysis) is a term used in software engineering to describe the testing of the dynamic behaviour of code. That is, dynamic analysis refers to the examination of the physical response from the system to variables that are not constant and change with time. In dynamic testing the software must actually be compiled and run. An error (either an exception or a spurious result) that arises at this stage is referred to as a run-time error. An exception, if handled properly in code may not even be seen by a user and may simply be logged or stepped over. However in the worst case the program may fail or even cause the whole operating system to become unstable or reboot. Spurious results may not in themselves result in program termination (although it will be appreciated that if relied upon they may cause termination at some later point). Typically a spurious error is an errors in logic, arithmetic or the like that causes an unexpected result (such as 1+1 equalling 5).
Unit Tests, Integration Tests, System Tests and Acceptance Tests are a few of the existing dynamic testing methodologies. Dynamic testing typically means testing based on specific test cases by execution of the test object or running programs.
It is not unusual for a run-time error to be missed by the test cases used in dynamic testing. Unless the developer has had the foresight to place error handling routines in code, a run-time error is often simply presented to a user as an obscure operating system level error with a memory reference that takes considerable expertise and time to identify its cause.
When a run-time error is encountered and is not easily identified, a common approach is to step through the code in a debugging environment to attempt to identify where the error occurred.
One semi-automated approach to this is called execution tracing. In execution tracing, tracing statements are added to the original code that logs the progress of execution through the code. For example, there may be a trace statement at the start of each routine and another at each branch in code. Each tracing statement writes data to a log as it is encountered in the code.
After the code has run, the log can be examined to identify the path traversed and give the developer more of an insight into where the code failed.
Execution tracing is a technique employed both as a debugging tool and also as a method of calculating the effectiveness of software tests (coverage analysis).
In order to provide an effective execution trace, existing techniques rely on fairly large amounts of information being recorded at trace points in order to allow the end user to observe whereabouts in the software the execution path went.
Given the size of a code base for a typical project or system, tracing is generally not feasible for a whole system and must be applied in a piecemeal fashion. Even then, the magnitude of logged data from tracing even selected modules can be significant and impair effective analysis of the problem. Full path tracing produces large amounts of output data, limiting the size of execution to which it can be applied.
While it is possible to reduce the amount of logged data by profiling techniques and code coverage techniques, these reduce the volume of output by recording summary information only. This is disadvantageous as the summary information gives only a vague idea of the area of a problem and also omits information on the full path traversed.